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ABSTRACT 



This document is an information resource for teachers based 
on the findings of the Assessment of Achievement Programme (AAP) , a major 
research program funded by The Scottish Office Education and Industry 
Department. This book presents information about the 1994 AAP mathematics 
survey and contains sections with sample survey items on student achievement; 
problem solving and inquiry; information handling; number, money, and 
measurement; and shape, position, and movement. Teachers can use the 
information presented here for personal study, staff meetings, and staff 
development. (Author/DDR) 
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WHAT IS FEEDBACK ? 



Feedback is an information resource for teachers, based on the findings of the 
Assessment of Achievement Programme (AAP),a major research programme 
funded by The Scottish Office Education and Industry Department This Feedback 
presents information about the I994AAP mathematics survey. 

Read Feedback to discover: 

□ the aspects of the 5-14 guidelines which were assessed. 

□ the key findings of the survey, against which you can compare the 
attainment of your own pupils. 

□ teaching issues based on the AAP survey findings. 

Use Feedback for: 

□ personal study - reflecting on the achievement of your own pupils, 
reviewing assessment of class work and assessment tasks. 

□ staff meetings - reviewing, collectively or in groups, teaching 
programmes in relation to the 5-14 attainment targets. 

□ staff development - considering effective teaching strategies, and helping 
student teachers as part of their professional training. 



Copyright SOEID, 1997 
First published May 1997 
ISBN 0 7480 5830 3 

Please note: you are free to photocopy or adapt Feedback for use in 
your institution. 
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Introduction 



The Assessment of Achievement Programme (AAP) 

The Assessment of Achievement Programme (AAP) was established by The Scottish 
Office Education and Industry Department (SOEID) in 1981 to monitor the perform- 
ance of pupils in Scottish schools in particular areas of the curriculum. Since 1983, 
there have been regular surveys in three core curricular areas - English language, 
mathematics and science. The main objectives of the AAP are to describe national 
levels of attainment and to provide evidence about changes in these levels over time. 
The surveys are intended to inform the SOEID, education authorities, teachers and 
other interested parties about the achievement of pupils, and to indicate ways of 
improving teaching and learning. 



The 1994 mathematics survey 

The fourth AAP survey of pupils’ attainment in mathematics was carried out in 1 994 
by a research team at the University of Strathclyde. This team also carried out the 
third survey in 1991. 

Samples of pupils at P4, P7 and S2 stages were selected to be representative of pupils 
in all mainstream schools. Over 9,000 pupils completed the assessments. 

The survey involved both written and practical mathematics tasks, including practical 
problem solving. 

Assessment was based on the curriculum defined in National Guidelines Mathematics 
5-/4. The main reporting categories adopted by the survey were Information handling 
Number ; money and measurement and Shape, position and movement. Reporting sub- 
categories were closely related to strands of the 5-14 guidelines. 



AAP and the 5-14 curriculum 

The assessment tasks were matched to levels A - E and ‘beyond level E’ as these are 
defined in the attainment targets of National Guidelines 5-/4. Members of the 1994 
AAP Mathematics Project Committee advised on the match of all new tasks to 5- 1 4 
levels and an independent consultant re-examined the match of tasks to levels in the 
1991 survey. 




What Our Pupils Achieved 



In general 

□ Attainment in many of the tasks was good. For detail about pupils’ per- 
formance, readers should consult the fuller report AAP Mathematics / 994 
published by SOEID. Inevitably, since the purpose of this document is to 
raise issues relevant to teaching, the examples chosen often illustrate poor 
performance or interesting patterns of error. 

□ Performance in Information handling was good. There was evidence of 
progression in problem-solving skills from P4 to S2, but in extended 
problem-solving tasks pupils preferred a random, rather than systematic, 
approach and were reluctant to show working or to handle materials. 

There were weaknesses in Number ; money and measurement and in Shape , 
position and movement , particularly at S2. 

□ Performance was higher if items were in a context which encouraged a 
method different from the common algorithm and lower when the 
context was unfamiliar or involved new terms or concepts. 

At P4 

□ Pupils’ performance was poorer than in 1991 in whole number in basic 
processes and applications. 

□ They found information from tables well but had more difficulty 
interpreting graphs and tables. 

□ Girls performed better than boys in Information handling , but there was 
little difference in the other attainment outcomes. 

At P7 

□ Pupils’ performance was poorer than in 1991 in whole number and decimals, 
fractions and percentages in both basic processes and applications. 

□ They found information from tables well but had more difficulty 
interpreting graphs and tables. 
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□ The performance of boys was better in Information handling and in Number ; 
money and measurement 

At S2 

□ Performance was poorer than in 1991 in whole number and decimals , 
fractions and percentages in basic processes and applications. 

□ S2 pupils did not perform significantly better in working with whole numbers 
than P7. 

□ Pupils found information from tables well and had less difficulty interpreting 
graphs and tables than P7 pupils. 

□ They performed well in line symmetry but had difficulty with rotational 
symmetry. 

□ The performance of girls was better in Information handling and the 
performance of boys was better in Shape, position and movement 



Problem Solving and Enquiry 



What was assessed? 

□ interpretation of the problem 

□ selection of a strategy 

□ carrying out the task 

□ reporting back 

The tasks were presented as short and extended response items in the written tests. 
There were also extended problem-solving tasks in the practical survey where pupils 
could use practical materials and report verbally to the field officer. 



Key findings 

□ Where the same task was used at more than one stage, there was a 
noticeable increase in the percentages of pupils successfully solving the 
problem. 

□ If a simple diagram would solve the problem, pupils preferred that 
strategy, otherwise they resorted to a written description. 

□ Most pupils used a ’trial and improvement' strategy in preference to a 
more systematic approach. 




Teaching issues 

What systematic strategies should we introduce pupils to, and at what 
stage? 

Should we teach a set of problem-solving strategies to pupils? If so, 
how do we teach these strategies? 

Do we give enough emphasis to insisting upon pupils setting down 
their working of a problem? 
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Example I 



A party of 32 people arrived to find seats at the tables. 
The round tables seat 6. The square tables seat 4. 
Each table has to be full. 



Q O 'D_OjO 

O D 






What is the least number of tables they need? 

They will need round tables and square tables. 



Correct solutions were produced by 32% of P4 pupils, increasing to 66% at P7. The 
incorrect answer of ’2 round tables and 5 square tables’ was given by a third of P4 
pupils and a tenth of P7 pupils. 

How do you think they arrived at that conclusion? 



Example 2 

In an extended task,P7 and S2 pupils were given a kilt pin and, from a manufacturer’s 
list of boxes, asked to select the smallest box which would hold the pin. 

About 40% at P7 and 60% at S2 were able to select and implement an appropriate 
strategy without prompting. Having found an apparent solution, few considered 
whether there might or might not be a better solution. 

Do these results simply reflect lack of experience with problem solving? 



Example 3 



A block of chocolate fudge is made in the shape of a cuboid. 




It measures 10 cm by 7 cm by 3 cm. 

1 Six of these blocks can be packed in a box measuring 
30 cm by 14 cm by 3 cm without any wasted space. 

Describe in words, or sketch, how this can be done. 

2 Six of the blocks can also be packed into a box measuring 
30 cm by 7 cm by 6 cm. 

Describe in words, or sketch, how this can be done. 

3 Find the dimensions of another box which will hold exactly 
6 blocks of this fudge. 



Most P7 pupils used a diagrammatic approach to the first two parts; 59% opted for a 
written description to part 3, but two thirds of these answers were unsatisfactory. 
The same problem produced similar approaches with S2 pupils but with somewhat 
better results. 

Why do you think pupils encountered difficulties with a written description? 



Information Handling 



What was assessed? 

The written tests assessed the strands 

□ display 

□ interpret 

The practical tests assessed the strands 

G collect 
G organise 
G display 



Key findings 

□ Performance was high overall compared to the other attainment 
outcomes. 

□ At P7, performance fell significantly when interpretation went beyond 
reading graphs and tables, with a smaller fall at S2. 

□ 20% of P4 pupils succeeding in drawing, unaided, a bar graph from data 
they had collected. 

G About half of P7 and S2 pupils successfully drew a bar graph with a non- 
unrtary scale from data they had collected. 




Teaching issues 

Do we provide sufficient progression in the amount and complexity of 
interpretation of data? What contexts might be introduced to achieve 
this? 

Do our programmes of study give sufficient emphasis to the use of 
computer spreadsheets and databases to increase experience in handling 
and interpreting data? Can you identify the progression in these which 
is needed for each school stage? 
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Example 4 



This graph shows the average monthly temperature throughout last year. 



Aunnann monthly temperature 




Month 



Ski-ing was possible when the temperature was below 5°C. 

Give the names of all the months when ski-ing was possible. 

Answer 

Level E 



Only 32% of P7 pupils managed to give the correct answer and only 54% of S2. 
despite, at both stages, the example being set within the theme of The Ski Trip. 

Can you identify any factors which may have caused pupils so much difficulty? 
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Number, Money and Measurement (I) 



What was assessed? 

The written tests assessed the strands 

□ range and type of numbers 

□ measure and estimate 

□ round numbers 

The practical tests assessed aspects of these strands, particularly estimation and 
rounding. 

Key findings 

□ P4 pupils handled place value with two-digit numbers fairly well but few 
succeeded when there were three digits. 

□ Many P7 pupils had difficulty with place value in decimals. 

□ Many P7 and S2 pupils were insecure with other than the simplest 
fractions and unfamiliar with other than the simplest whole number 
percentages. 

□ At P7 and S2, converting from one metric unit to another gave the 
greatest difficulty when the conversion factor was 1 000. 

□ In the practical tests, on average less than 10% at P7 and 20% at S2 
demonstrated adequate skill in estimating the answers to calculations. 




Teaching issues 

Is sufficient emphasis given in teaching programmes to understanding 
place value? 



Would introducing percentages through decimal, rather than common, 
fractions make the concepts accessible to more pupils? 





What strategies could be adopted to improve pupils’ abilities in 
estimating answers? 
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Example 5 



Complete this table. 

The first row has already been filled in for you. 



Decimals 


Fractions 


0.50 


1 

2 


0.10 




0.15 





Level D 



Most P7 pupils correctly identified 0. 1 0 as a tenth, but the success rate fell from 72% 
to only 16% with O.IS; the answers one-fifteenth and one quarter were the most 
common incorrect responses for the latter question. 

How might pupils have arrived at these incorrect answers ? 



Example 6 




Only 22% of P7 pupils and 38% of S2 gave the correct answer; this compares to 3 1 % 
at P7 and 43% at S2 in 1 991. The most common incorrect answer at both stages 
was 32. 

Why does such a simple example give so much difficulty? 






Number, Money and Measurement (2) 



What was assessed? 

The written tests assessed the strands 

□ patterns and sequences 

□ functions and equations 



Key findings • ; 

□ Most P4 and P7 pupils were able to complete number sequences with I 

simple rules. • j 

! 

□ Many S2 pupils had difficulty in constructing formulae and in., 
substitution into algebraic expressions. 

□ Solution of equations by S2 pupils was well done, but performance fell 
sharply as equations became more complex. 




Teaching issues 

Are algebraic ideas introduced early enough to pupils? If not, should 
we review the stage at which algebra is begun? 

Is there enough direct teaching of efficient methods of solving equations? 
Are pupils given enough practice in solving equations? 






Example 7 



Brian and Karen arrange their books on a shelf. Numbers on books are 
linked in a simple way. 

Write the missing number on the book. 




Level D 



Despite the difficulty of the pattern of this sequence, there was a high performance 
by P4 pupils with 42% successful in completing it. 



Example 8 

Solve for x. 

3x + 7= 16 

Answer 

Level E 



Performance by S2 pupils was slightly better than in 1991 with 63% able to find the 
correct solution. 

Would you consider this a good level of performance? What strategies might 
pupils have used to find the solution? 




Example 9 

A parcel weighing N kg costs 
(10 N + 30) pence to deliver. 

What is the cost of delivering a 7 kg parcel? 

Answer 

Level E 



Only around one third of S2 pupils correctly solved this problem and around one 
third made no attempt. 

Why does this problem give so much difficulty? 
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Number, Money and Measurement (3) 



What was assessed? 

The written and practical tests assessed the strands 

3 add and subtract 
□ multiply and divide 
3 money 



Key findings 

□ At P4, performance on basic processes was significantly lower than in 
1991. 

□ P7 pupils* performance in working with whole numbers was much the 
same as in 1991, but that of S2 pupils was significantly lower than in 
1991. 

□ Skill in using calculators, at all stages, was very high. 

□ When pupils had to decide which operation to select for a calculation, 
performance was significantly lower than where the operation was 
specified. 




Teaching issue 

Should more emphasis be given to performing basic calculations without 
the aid of a calculator? 




. ,v- .• 
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Example 10 

Subtract 

82-36 
Answer _ 
Level B 



P4 performance fell from 59% to 43% between the 1991 and 1994 surveys .The answer 
of 54 was given by 1 7% of pupils. 

Can you explain the likely reasoning behind the incorrect answer? Why is 
performance so low on this example? 



V * x .y; 







Example 1 1 

John is saving up for a new bicycle which costs £120. 

He has already saved £72 and has a part-time job from which he can save 
£7 each week. 




How many complete weeks will it be before he can buy the bike? 

weeks 

Level D 



42% of P7 and 54% of S2 gave the correct answer.The most common mistake at 
both stages was to round down to 6 wgeks. 

Why is there so little improvement in performance from PI to S2? 
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Number, Money and Measurement (4) 



What was assessed? 



The written and practical tests assessed the strand 
□ decimals, fractions and percentages 



□ Many P7 and S2 pupils had difficulty multiplying and dividing decimals by 



Key findings 





pupils considerable difficulty. 



□ Performance on basic processes was significantly lower at P7 and S2 
than in 1991. 

□ In the practical survey, P7 and S2 pupils were very skilled at using 
calculators for calculations involving decimals. However performance 
was much lower if rounding was involved. 



Teaching issues 

Should more use be made of concrete materials to give understanding 
of the effect on place value of multiplying and dividing by multiples of 



Should the relationship between percentages and decimal rather than 
vulgar fractions be given more emphasis? 




10 ? 
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Example 12 



Multiply 

35*2 x 100 
Answer 
Level D 



Only 36% of P7 pupils could answer this question correctly, a proportion which has 
remained low over successive surveys. A quarter gave 352 as the answer. 

Example 13 

Calculate 

2 A x 25 

Answer 

Level D 



Only 39% of S2 pupils were correct and around one fifth made no attemptThe success 
rate was lower than in 1991 when 47% were correct. 

Why does such a straightforward example give difficulty? 



Number, Money and Measurement (5) 



What was assessed? 

The written and practical tests assessed the strands 

3 perimeter, formulae, scales 

□ measure and estimate 

□ time 

□ money 



Key findings 

□ At all stages, performance on applications of time was high. 

□ S2 performance was higher than P7 on many tasks involving applica- 
tions of ratio, formulae and scale. 

□ Performance in aspects of perimeter and area was disappointing and 
some P4 pupils seemed unfamiliar with the term perimeter. 

□ Many P4 and P7 pupils had difficulty finding the actual weight of an 
object but more could put objects in order of weight using a two-pan 
balance. 

□ Few pupils could calculate correctly a time interval which crossed a 
change in hour. 

□ Most S2 pupils could convert a 24-hour clock time into am/pm 
notation. 




Teaching issue 

How might performance in perimeter and area be improved? With 
more use of practical activity? By introducing aspects of these at an 
earlier stage? 
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Example 14 



Calculate the perimeter of the rectangle. 




4 7 metres ► 








4 metres 

4 


Answer 


metres 


Level D 







Only 54% of P7 pupils calculated the perimeter correctly, although this was an 
improvement on the 42% correct in 1991. Performance at S2 was the same. Nearly 
30% at P7 and at S2 gave the answer as 28 metres. 

Can you identify the main confusion in pupils * minds? 



Example IS 

What is the volume of this cuboid in cubic centimetres? 



Answer 
Level E 



Less than a quarter of P7 pupils could apply the correct formula to find the volume 
successfully. 

Speculate on the kinds of incorrect answers which would have been given. 



nr 



1 0cm 



5cm 
1 0cm 



crrr 
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Shape, Position and Movement (I) 



What was assessed? 

The written and practical tests assessed the strands 

□ range of shapes 

□ symmetry 



Key findings ; 

□ Most P4 pupils could recognise and name common 2D and, 3D shapes. 

r . i.\/ > v r . ... . * - ’ * ■ i'll 

□ At S2, over 60% of pupils could work with the relationship between 

_ radius arid diameter. . . •• " : ! 

□ Most P7 pupils could complete symmetrical shapes and knew about. ; 

lines of symmetry. • 




Teaching issue 

Given good performance by pupils in this aspect of mathematics, should 
we review the pace at which the concepts are introduced? 
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Example 16 



Draw the lines of symmetry in the following rectangle. 



Level D 



52% of S2 pupils showed both lines of symmetry but 29% included the diagonals as 
lines of symmetry. It appears that if the rectangle is much longer and thinner pupils 
are less likely to make this mistake. 
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Shape, Position and Movement (2) 



What was assessed? 

The written and practical tests assessed the strands 

□ position and movement 

□ angle 



Key findings 

□ Most P4 pupils could use simple map-type co-ordinates but had 
difficulty with compass bearings. 

□ Most P7 and S2 pupils could use map-type and Cartesian co-ordinates 
well. 

□ Most P7 and S2 pupils successfully read a grid reference from a map but 
were less successful at giving directions. 




Teaching issues 

What practical activities using a compass might be introduced to 
teaching programmes? 

Would more use of computer packages involving movement assist 
understanding of this area of mathematics? 
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Example 17 



The house is north of the river and east of the road. 




Complete the sentence: 

The school is of the river and of the road. 

Level B 

Fewer than a third of P4 pupils have answered this correctly in each of the last three 
surveys. 'South and east' and 'north and east’ were common answers. 

Example 18 

A boat changes direction from East through South to South West. 

Through what angle has it turned? 

Answer ° 

Level D 



Only one third of S2 pupils could give the correct answer, similar to the levels of 
performance in the previous two surveys. 



Related publications 

The full report on the 1994 AAP Mathematics survey is available from the 
research team. Copies can be obtained by sending a cheque for £19.50 (payable 
to the University of Strathclyde) to Department of MSTE. Faculty of Education 
(Jordanhill), 76 Southbrae Drive, Glasgow G 1 3 l PR 

A limited number of additional copies of the summary report of the survey, AAP 
Mathematics 1994, can be obtained by writing to the SOEID Dissemination 
Officer, do SCRE, 15 St John Street, Edinburgh EH8 8JR. A copy for electronic 
downloading is available on the Educational Research in Scotland World Wide Web 
Server, accessible through the Internet and JANET (http://www.ed.ac.uk/-riu) 

As a result of the concerns raised in the AAP mathematics report, HM Inspectors 
decided to investigate good practice in the teaching of mathematics, particularly 
number, and to publish a report giving recommendations designed to improve 
performance in both primary and secondary schools. Shortly after this inspection 
task was begun, the Third International Mathematics and Science Study (TIMSS) 
report was published. The findings of this report reinforced concerns about 
Scottish standards in mathematics in the early stages of secondary school. TIMSS 
found that Scottish 1 3 year olds performed poorly in mathematics compared to 
many other countries — Scotland was 26th out of 4 1 countries, an uncomfortably 
low position. HM Inspectors have taken steps to identify features leading to the 
strong performance of pupils in some of the highest scoring countries. Their 
conclusions, along with recommendations deriving from inspection of good 
practice in Scottish schools, will shortly be published in an HMI report on the 
teaching of mathematics. 
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